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(57) Abstract: The process is characterized in that, the zoomed image being defined by its horizontal and vertical homothety ratio 
with respect to the coded image (1, 7), it performs the following steps: calculation of the horizontal and vertical sizes of an initial 
display rectangle (8) from which would be obtained the sizes of the target display rectangle (2) as defined by the profile and level 
information by applying the horizontal and vertical homothety ratios; storage in the display horizontal size and display vertical size 
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CODING PROCESS AND DEVICE FOR THE DISPLAYING OF A ZOOMED MPEG2 CODED IMAGE 



The invention relates to a coding and decoding process and 
5 device compatible with the MPEG2 standard for displaying a zoom of a 
transmitted image. Performing a zoom of an image consists in enlarging or 
reducing a part of this image. It therefore involves a modification of the 
horizontal and vertical dimensions of the image part selected for display in 
zoom mode. 

10 Nowadays, applications based on interactivity necessitate the 

ability to view images in reduced or magnified form. For example, electronic 
program guides utilize images of reduced sizes so as to allow display on the 
screen of ancillary information in OSD form (standing for On Screen Display). 
Magnification may also be necessary in the case where one wishes to focus 

1 5 the display on a part of the source image. 

The process proposed can be used by any type of apparatus 
utilizing a stream of data coded according to the MPEG2 standard, for 
example a decoder, a DVD reader/recorder, an Internet box, etc. The 
transmission of the data may be transmission by satellite, by cable, terrestrial 

20 transmission, etc. 

MPEG2 compression is not an object-based processing and the 
decoded image displayed is generally a copy of the coded source image. To 
achieve a specific display in zoom mode, it is therefore necessary to code 
the source image such as it must be displayed, that is to say after having 

25 performed the zoom on this source image. Thus, the source image, 
originating for example from a studio providing a video sequence, is 
processed before the coding and transmission of the video data. The zoom 
operations are performed on this source image and it is the new "zoomed" 
image which is coded like the other images of the video sequence. The 

30 drawback of such a process stems from the lack of flexibility which it 
imposes. The zooms to be carried out must be decided on at the time of 
coding. These zooms can then no longer be modified on receipt of the data, 
at the decoder level, for example as a function of data or parameters 
selected by the viewer. 

35 

The aim of the invention is to alleviate the aforesaid drawbacks. 
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Accordingly, the subject of the invention is a coding process 
allowing the display of a zoom of an image coded according to the MPEG2 
standard, the zoomed image being defined by its horizontal and vertical 
homothety ratio with respect to the coded image, characterized in that it 
5 performs the following steps: 

- calculation of the horizontal and vertical sizes of an initial display 
rectangle from which would be obtained the sizes of the target display 
rectangle, as defined by the profile and level information by applying the 
horizontal and vertical homothety ratios, 

10 - storage in the display_horizontal_size and display_vertical_size 

fields of an MPEG2 video data stream of the horizontal and vertical sizes of 
the initial display rectangle. 

According to a variant, the zoomed image being defined by its location in the 
displayed image in the event of a diminishment or by the part of the coded 
15 image to be displayed in the event of a magnification, the above process is 
characterized in that it also performs the following steps: 

- calculation of the corresponding position of the centre of the 
initial display rectangle with respect to the centre of the coded image so as to 
define a horizontal and vertical offset, 

20 - storage of an item of information defining the horizontal and 

vertical offset in the frame_centre_horizontal_offset and 
frame_centre_vertical_offset fields of the MPEG2 video data stream. 

Its subject is also a coding device for implementing the above 
coding process, characterized in that it comprises a syntax modifier linked to 

25 an MPEG coder for modifying the display_horizontal_size and 
display_vertical_size fields and the frame_centre_horizontal_offset and 
frame_centre_vertical_offset fields of the MPEG2 data stream originating 
from the coder, respectively as a function of the amplitude and of the position 
of the desired zoom. 

30 According to a variant, the above device is characterized in that it 

comprises an application synchronizer linked to the coder and to the syntax 
modifier for controlling the syntax modification between two temporal code 
instants tcO and tc1 of the MPEG2 data stream corresponding to the images 
for which the zoom function is desired. 

35 The idea of the invention is to transport a certain number of 

parameters relating to the zoom of the image in fields defined in the syntax 
of the binary stream of video data coded according to the video MPEG2 
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standard. The fields are chosen from among the extensions provided for in 
the standard. Thus, the coder can dynamically manage the parameters 
relating to the zoom, that is to say the dimension of the image to be viewed, 
its location in the image displayed, the part of the coded image to be viewed 
5 in zoom mode, doing so without changing the coding parameters such as the 
horizontal dimension and vertical dimension of the coded image. 

The various existing fields, in the video MPEG2 standard 
(ISO/CEI Standard 13818-2), relating to the size of the images to be coded 

10 and to be displayed are recalled hereinbelow: 

The dimension of the coded image or more exactly of the 
displayable image is defined in the sequence header of the MPEG2 data 
stream (paragraph 6.3.3 of the video MPEG2 standard) and optionally in the 
sequence extension (paragraph 6.3.5 of the video MPEG2 standard). 

15 These are the horizontal_size, vertical_size fields and the 

horizontal_size_value, vertical_size_value and horizontal_size_extension, 
vertical_size_extension fields to which the first fields make reference. These 
fields in fact define the width of the displayable part of the luminance 
component in pels (pixels) and the height, in terms of number of lines, of the 

20 displayable part of the luminance component of the complete image (two- 
frame) in pels. The displayable image part, which is the image part 
containing the video and which does not therefore necessarily comprise an 
integer number of macroblocks, is aligned on the left edge and the upper 
edge of the coded images. 

25 The dimension of the image displayed is, for its part, defined in 

the sequence_display_extension (paragraph 6.3.6 of the video MPEG2 
standard). It will be noted that this extension does not define the display 
procedure. The information of this extension has no relevance to the 
decoding procedure and can be neglected by the decoders in accordance 

30 with the MPEG2 specification. The two structures, displayJiorizontal_size 
and display_verticaLsize, together define a rectangle which may be 
regarded as being the active region of the envisaged display. If this display 
rectangle is smaller than the coded two-frame image, then the display 
procedure may be expected to display only a part of this image. Conversely, 

35 if the display rectangle is larger than the coded two-frame image, the display 
procedure may be expected to display the reconstructed two-frame images 
only on a part rather than the whole of the display device. 
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The MPEG2 standard envisages a picture_display_extension 
structure, the subject of paragraph 6.3.12 of the ISO/CEI document 13818-2. 
The picture_disp!ay_extension makes it possible to offset, picture by picture, 
the position of the display rectangle, whose sizes are specified by the 
5 previously described "sequence_display_extension" structure. One 
application relates to the refraining of images. The frame_centre_horizontal_ 
offset and frame_centre_vertical_offset fields indicate the horizontal and 
vertical offset of the centre of the reconstructed image with respect to the 
centre of the display rectangle. The image centre offsets may serve to 

10 implement the full-screen reframing process, by which a rectangular zone 
corresponding to the full screen is defined and is shifted panoramically over 
the entire reconstructed image. This process is used in particular to go from 
an image coded in the 16/9 format to an image displayed in the 4/3 format. 
The zone defined by the 4/3 format can thus contain the most interesting part 

15 of the original 1 6/9 format image. 

It will be noted that the fields which are defined in the standard in 
respect of a sequence (sequence header, sequence display extension) may 
in fact be transmitted per picture (cf. reserved values in Table 6.2 of the 
"identification codes for extension triggering codes" standard). 

20 Amplitude of the zoom. 

The idea proposed here is to regard the information contained in 
the u display_horizontal_size" and "display_vertical_size" fields as defining the 
amplitude of a zoom. 

Instead of defining, with the aid of these fields, the dimension of 

25 the display rectangle or image displayed, as proposed by the MPEG2 
standard, one defines a dimension of an "initial display rectangle" or original 
image from which the zoom would be performed. The dimension of this initial 
display rectangle is such that the zoom applied to this rectangle gives an 
image whose dimensions are those of the target display rectangle, that is to 

30 say of the dimension of the screen, as defined by the choice of the profile 
and level by the coder. 

Thus, an initial display rectangle of dimensions greater than those 
of the target display rectangle corresponds to a diminution of the image and 
a rectangle of smaller dimensions corresponds to a magnification. 

35 The amplitude of the zoom is therefore obtained by calculating the 

ratio of the size of the target display rectangle, which is deduced from the 
level and profile information, to the size of the initial display rectangle, which 



WO 02/47393 



5 



PCT/EP01/13285 



is transported in the "display_horizontal_size" and "display_vertical_size" 
fields. Of course, these latter fields should not be used for the transmission 
of redundant information concerning the conversion of the 16/9 format into 
the 4/3 format. 

5 An exemplary use of the fields described above is given below 

and shown schematically in Figure 1 . 

Let us consider a decoded or reconstructed image referenced 1 in 
the figure, whose dimension is 480 pixels by 576 lines (this is in fact the 
dimension of the image encoded at the level of the MPEG coder), a target 
10 display rectangle or screen dimension referenced 2 of 720 pixels by 
576 lines. 

The following table indicates the various zoom amplitudes as a 
function of the information transported in the "display_horizontal_size" and 
"display_vertical_size" fields. 

15 The values h_zoom and v_zoom correspond to the horizontal and 

vertical homotheties. A zoom amplitude of less than 1 means that the 
dimension of the image obtained relative to the reconstructed image, with 
respect to the dimension of the coded image, is less than 1 and a zoom 
amplitude of greater than 1 indicates that the dimension of the image 

20 obtained with respect to that of the coded image is greater than 1. If this 
value is equal to 1 , then the image obtained is the reconstructed image. 



"Displayjhorizontal 
jsize" 

(pixels) 


"Display_vertical_ 
size" 

(lines) 


Target 
display 


Amplitude of the zoom 
h_zoom, v_zoom 


1 440 


1 152 


720 x 576 


h_zoom = 720/1 440 = 0.50 
v_zoom = 576/1 152 = 0.50 1 
*Note 1 


2 880 


2 304 


720 x 576 


h_zoom = 720/2 880 = 0.25 
v_zoom = 576/2 304 = 0.25 
*Note 1 


720 


576 


720 x 576 


h_zoom = 1 
v_zoom = 1 


480 


576 


720 x 576 


h_zoom = 720/480 = 1.50 
v_zoom = 576/576 = 1.00 
* Note 3 
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360 


288 


720 x 576 


h_zoom = 720/360 = 2.00 








v_zoom = 576/288 = 2.00 








*Note 2 


180 


144 


720 x 576 


h_zoom = 720/180 = 4.00 
v_zoom = 576/144 = 4.00 
*Note 2 



Note 1: Diminution of the image 
Note 2: Magnification of the image 

Note 3: The image obtained from the coded image, the dimensions of which 
5 are greater than those of the coded image, fills the screen. 



The target display rectangle (or target display) deduced from the 
profile and level information and from the image format information (16/9) is 
greater than the size of the coded image. 
10 The images referenced 3, 4, 5, 6 correspond respectively to the 

values: 

h_zoom = v_zoom = 0.50 
h_zoom = v_zoom = 1 .00 
h_zoom = 1.50, v_zoom = 1.00 
15 h zoom = v zoom = 2.00 



The display_horizontal_size and display_vertical_size fields are 
14-bit fields. The maximum decimal value, equal to 16 383, corresponds to a 
maximum diminution of 720/16 383 horizontally and 576/16 383 vertically, in 
20 our example. 

Location of the zoomed image. 

The frame_centre_horizontal_offset and frame_centre_vertical_ 
offset fields of the image display extension structure are utilized to transport 
the zoomed image location information. Same corresponds to the offset of 
25 the centre of the coded image with respect to the centre of the initial display 
rectangle. 

The centre of the coded image (in fact of the displayable image) is 
the centre of the region defined by the u horizontal_size" and u vertical_size" 
fields of the sequence header. 
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The centre of the initial display rectangle is the centre of the 
region defined by the "display_horizontal_size" and "display_vertical_size" 
fields, as indicated above. 

5 Figure 2 gives the example of an image diminution and represents 

an initial display rectangle 8 in which lies the coded image 7 corresponding 
to the coded image 1. The horizontal and vertical arrows depict the 
horizontal and vertical offset of the centre of the coded image with respect to 
the centre of the initial display rectangle. 

10 Let us consider for example a decoded or reconstructed image of size 
480 pixels x 576 lines, the "display_horizontal_size" and "display_vertical_ 
size" fields respectively transporting the values 1 440 (pixels) and 
1 152 (lines) and a size of the target display rectangle of 720 (pixels) x 
576 (lines). 

1 5 h_zoom = 720 /1 440 = 0.50 
v_zoom = 576/1 152 = 0.50 

If one wishes to position a zoomed image in the upper left part of the initial 
display rectangle, for example at a distance of 400 pixels horizontally and 

20 300 lines vertically, from the upper left corner of the initial display rectangle, 
the offset information will have the following values: 
frame _centre _horizontal _offset = (400 - 1 440 / 2) x 16 
frame _centre ..vertical _offset = (300 - 1 1 52 / 2) x 1 6 
The two values are negative in accordance with the MPEG norm. 

25 The values are multiplied by 16 since the offsets are specified in units of 
1/16thofapel. 

The values 400 and 300 correspond to the pre-zoom values of the image. 
These values must be corrected for the zoom amplitude if one wishes to 
define the positioning of the zoomed image in the image actually viewed on 
30 the screen. 

In the case of an image magnification, the position of the initial display 
rectangle (8) in the coded image (7) defines the zone of the magnified coded 
image which one wishes to view. 

35 A particular implementation of the invention relates to the dynamic 

formatting of the video from an interactive application. 
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Figure 3 represents a simplified diagram of a device for such an 
implementation. 

The various video source images are received by MPEG2 video 
coders 9 t to 9 n . The images are coded according to a constant image 
5 dimension, for example 720 pixels by 576 lines. Video syntax modification 
circuits 10j to 10 n which are or are not part of the video coders provide 
interfaces for modifying fields of the MPEG2 video syntax for the associated 
coders i to n. 

A toolbox for the application 12 is composed of: 
10 - an application injector, for example in the Open TV, Media 

Highway, Java format, these being registered trade marks, 

- an application synchronizer which synchronizes the application 
with the video for the execution of this application according to the time code 
information of the MPEG2 elementary stream. 
15 This toolbox 12 receives information originating from the various 

coders i to n and transmits data for modifying the fields to the syntax 
modification circuits 10 ( to 10 n . 

It is linked to a multiplexing circuit 1 1 for transmitting private data 
with their PID (Packet IDentifier), in the form of sections. 
20 The various coders 9 1 to 9 n , through the video syntax modification 

circuits 10 ( to 10 n as the case may be, are also linked to the multiplexing 
circuit 11 which, on the basis of the various elementary streams and private 
data, provides a transport stream. 

This transport stream is then utilized so as to be transmitted to an 
25 IRD receiver (standing for Integrated Receiver/Decoder). This receiver, not 
represented in the figure, comprises, among other things, a decoder for 
decoding the audio/video data and an application engine for implementing 
the application package. It may also be a simple means of digital storage 
such as a hard disk, DVD or the like for subsequent utilization of the 
30 recorded data. 

The syntax modification circuit 10 modifies basic fields of the 
MPEG2 video syntax allowing image zooms to be carried out according to 
the process described above. 

In the video stream, these fields are modified over a time space 
35 corresponding to the duration of execution of the application. The time codes 
relating to the initial and final images between which the application is to be 
implemented at receiver level are transported by the application (private 
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data). This is "start time code" information (tcO) and "stop time code" 
information (td) which therefore correspond to the start and to the end of the 
application. At the receiver, the application engine extracts the "time code" 
information for the various data streams and the application is triggered, for a 
5 given stream, from the reading of the time code corresponding to tcO, up to 
the instant td where it stops. Between these two instants, the display is 
performed in zoom mode. 

The application synchronizer 12 receives the time codes of the 
various coders. It modifies, by way of the syntax modification circuits, the 

10 fields of the MPEG2 video syntax, between start time code instant which 
corresponds to the starting of the application and stop time code instant 
which corresponds to the end of the application, thus making it possible to 
reduce (or increase) the size of the video image according to the process 
described above, for images lying between these instants. 

15 This "start time code" and "stop time code" info can originate from 

outside data. It is, for example, obtained from the identification of an image 
sequence during which one wishes to be able to perform a zoom, which 
sequence is defined by way of these time codes. 

In one example, the application supplements the displaying of the 

20 diminished image with a display in interactive OSD mode, by inserting 
information into the image. This interactive mode makes it possible to return 
to the normal mode, by interrupting the application. 

Figure 4 represents, on a first line, the image 13 as constructed by 
the coder between the instants tcO and td during which the application runs, 

25 . as well as the images preceding the starting of the application and following 
the end of the application. On a second line are represented the image 14 as 
displayed by the decoder when the application runs, the image preceding 
and the image following the application. 

Between the instants tcO and td, the application, at the receiver, 

30 runs and reduces the size of the image as a function of the data read from 
the fields defined above. 

The MPEG decoder receives the data stream which it decodes in 
a conventional manner. In particular, the images for which a zoom operation 
35 is possible on account of the filling of the fields relating to the amplitude of 
the zoom and to the position of the zoomed image are decoded so as to 
provide firstly a reconstructed image whose size is indicated in the 
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"horizontal size" and "vertical size" fields. This reconstructed image is then 
oversampled or undersampled and then filtered as a function of the 
amplitude of the zoom so as to obtain the dimension of the image which one 
desires to display. The position of the imagette (diminishment) or the 
5 selection of the image part to be viewed (magnification) are calculated as a 
function of the data relating to the offset. 

An exemplary use may be the insertion of an advert. The image 
dimension is reduced so as to show up some text. Another example relates 
10 to a program guide. In the prior art, the viewer makes a choice on an array 
constructed at the coder level and the selection of an imagette is manifested 
by zapping to another video service (elementary video stream). 

In the invention, the selecting of a program, for example from an 
OSD display, triggers the starting of the application for the MPEG data 
15 stream corresponding to the selected program. The channel selected is then 
displayed "live" (in real time) in a window. 



\ 
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CLAIMS 



5 1 Coding process allowing the display of a zoom of an image 

coded according to the MPEG2 standard, the zoomed image being defined 
by its horizontal and vertical homothety ratio with respect to the coded image 
(1, 7), characterized in that it performs the following steps: 

- calculation of the horizontal and vertical sizes of an initial display 
10 rectangle (8) from which would be obtained the sizes of the target display 

rectangle (2) as defined by the profile and level information by applying the 
horizontal and vertical homothety ratios, 

- storage in the display_horizontal_size and display_vertical_size 
fields of an MPEG2 video data stream of the horizontal and vertical sizes of 

15 the initial display rectangle. 



2 Process according to Claim 1 , the zoomed image being defined 
by its location in the displayed image in the event of a diminishment or by the 

20 part of the coded image to be displayed in the event of a magnification, 
characterized in that it performs the following steps: 

- calculation of the corresponding position of the centre of the 
initial display rectangle (8) with respect to the centre of the coded image (7) 
so as to define a horizontal and vertical offset, 

25 - storage of an item of information defining the horizontal and 

vertical offset in the frame_centre_horizontal_offset and 
frame centre vertical offset fields of the MPEG2 video data stream. 



30 3 Process according to Claim 1 , characterized in that the syntax 

modification of the stream corresponding to the storage of the values in the 
fields is performed between two temporal code instants tcO and tc1 of the 
MPEG2 data stream corresponding to the images for which the zoom 
function is desired. 



35 
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4 Coding device for implementing the coding process according to 
Claim 2, characterized in that it comprises a syntax modifier (12) linked to an 
MPEG coder (9) for modifying the display_horizontal_size and 
display_vertical_size fields and the frame_centre_horizontal_offset and 

5 frame_centre_vertical_offset fields of the MPEG2 data stream originating 
from the coder, respectively as a function of the amplitude and of the position 
of the desired zoom. 

5 Device according to Claim 4, characterized in that it comprises 
10 an application synchronizer (12) linked to the coder (9) and to the syntax 

modifier (10) for controlling the syntax modification between two temporal 
code instants tcO and td of the MPEG2 data stream corresponding to the 
images for which the zoom function is desired. 

15 6 Process for decoding an MPEG2 data stream, characterized in 

that it performs the following operations: 

- reading of the data in the frame_centre_horizontal_offset and 
frame_centre_vertical_offset fields of the MPEG2 video data stream, 

- reading of the data in the display_horizontal_size and 
20 dispIay_verticaLsize fields of an MPEG2 video data stream, 

- calculation of the homothety ratio of the zoom as a function of 
the chosen horizontal and vertical sizes of the zoomed image corresponding 
to these size data, 

- oversampling or undersampling or filtering or interpolation of the 
25 coded image received as a function of this homothety ratio, 

- positioning in the image to be displayed, of the image thus 
sampled as a function of the data relating to the offset. 

7 Receiver comprising a decoder of a stream of MPEG2 data 
30 coded according to the process of Claim 3, an interactive application engine, 
characterized in that the application engine implements the application 
relating to the zoom function in respect of the images (14) whose temporal 
codes lie between the values tcO and td. 
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